Measuring differential item and test functioning across academic disciplines
نویسنده
چکیده
Differential item functioning (DIF) is when a test item favors or hinders a characteristic exhibited by group members of a test-taking population. DIF analyses are statistical procedures used to determine to what extent the content of an item affects the item endorsement of sub-groups of test-takers. If DIF is found for many items on the test, the final test scores do not represent the same measurement across groups in the population of test-takers. This is known as differential test functioning (DTF). DTF is of particular concern in tertiary level language tests, where test-takers often differ in academic discipline. This study examined the DIF and DTF of an in-house developed assessment designed to measure how well first year students of five academic disciplines achieved material over the course of a year of English language study. The DIF and DTF tests were performed using Rasch analysis, which controls for ability across groups, ensuring that items are only flagged if groups of test-takers of the same ability levels exhibit a significantly different probability of endorsing the item. The current analysis outlines the process for checking for DIF and DTF and finds that even though DTF is unlikely, there were several items that favored and hindered some majors. Recommendations for modification of items are made and the importance of establishing a process to check for DTF and DIF, especially when the test-takers are from different disciplines of study, is discussed.
منابع مشابه
Gender-based DIF across the Subject Area: A Study of the Iranian National University Entrance Exam
This study aimed at investigating differential item functioning (DIF) on the Special English Test of the Iranian National University Entrance Exam (INUEE). The effect of gender and subject area was taken into account. The study utilized one-parameter IRT model with a sample of 36000 students who sat for the INUEE Special English Test in 2004 and/or 2005. The findings confirmed the presence of D...
متن کاملSelecting the Best Fit Model in Cognitive Diagnostic Assessment: Differential Item Functioning Detection in the Reading Comprehension of the PhD Nationwide Admission Test
This study was an attemptto provide detailed information of the strengths and weaknesses of test takers‟ real ability through cognitive diagnostic assessment, and to detect differential item functioning in each test item. The rationale for using CDA was that it estimates an item‟s discrimination power, whereas clas- sical test theory or item response theory depicts between rather within item mu...
متن کاملDifferential Item Functioning (DIF) in Terms of Gender in the Reading Comprehension Subtest of a High-Stakes Test
Validation is an important enterprise especially when a test is a high stakes one. Demographic variables like gender and field of study can affect test results and interpretations. Differential Item Functioning (DIF) is a way to make sure that a test does not favor one group of test takers over the others. This study investigated DIF in terms of gender in the reading comprehension subtest (35 i...
متن کاملDifferential Item Functioning and Unidimensionality in the Pearson Test of English Academic
Since the Pearson Test of English Academic (PTE Academic) was designed to assess skill differences among test-takers at all points along the ability continuum, rather than to determine cutoff scores, it is important to examine the extent to which the instrument assesses what it is intended to measure (validity) as well as the extent to which the test is consistent (reliability) in measuring ELL...
متن کاملA confirmatory study of Differential Item Functioning on EFL reading comprehension
The present study aimed at investigating DIF sources on an EFL reading comprehension test. Accordingly, 2 DIF detection methods, logistic regression (LR) and item response theory (IRT), were used to flag emergent DIF of 203 (110 females & 93 males) Iranian EFL examinees’ performance on a reading comprehension test. Seven hypothetical DIF sources were examin...
متن کامل